Evidence of Analytical and Integrative Thinking
Over the past two and a half years, I have noticed a transformation within myself from classroom teacher to researcher. When I first entered the PhD program, my experience had been as an elementary special education teacher and as a mentor to first-year special education teachers. As I began my work as a graduate research assistant (GRA) that first semester, even before beginning my classes, I quickly realized that the demands of research were not the same as the demands of being a teacher. It was a difficult shift for me to make. I loved being a teacher and felt that what I did was very important. In the role of researcher, I felt my skills were being limited and that I was not able to provide the best possible education that I knew how to provide. Over time, through my experiences as a GRA and through my coursework, I realized the value of the researcher role and the great impact it can have on student learning. Now I find myself thinking like a researcher, even when discussing instruction outside of research contexts.
I have gained a tremendous amount of knowledge about research and teaching through my experiences in the PhD program and as an instructor. Additionally, I have participated in service activities within the College of Education and Human Development and beyond. Here I attempt to summarize my learning in the areas of research, teaching, and service, and provide links to documents that demonstrate this growth over time.
Research
Coursework
One of the recurring themes of my very first class in the PhD program, EDUC 800: Ways of Knowing, was that what you see – what you know – depends on how you look at it. In research, each methodology reveals something, but it also conceals something. One of our assignments for the course challenged us to explore a way of knowing different from our own as a way to increase our awareness of these differences. I chose to explore critical multicultural inquiry as a way of knowing. That exercise certainly broadened my perspective. In another course I took that first semester, EDRS 810: Problems and Methods in Education Research, we discussed how the research questions you chose determine the method that you will use to collect and analyze data. For that class, I designed studies to look at one topic in two ways. I investigated literacy instruction for individuals with moderate and severe disabilities by designing a quantitative intervention study, and by designing a qualitative case study of students’ literacy experiences.In subsequent classes, I gained more practice writing different types of studies. For example, in EDSE 842: Applications of Research Methodology in Special Education, I wrote four methods sections – an experimental design, a survey design, a single-subject design, and a qualitative design. I also wrote a single-subject research proposal for EDEP 654: Learning, Motivation, and Self-regulation, modeled on the writing intervention studies I had worked on as a GRA.
In my research methods courses, I had the opportunity to advance my skills further by not only designing studies, but also collecting and analyzing data. My research study for EDRS 812: Qualitative Methods in Education Research focused on how teachers of students with severe emotional and behavioral disabilities (EBD) establish positive, trusting relationships with their students. For that project, I interviewed three teachers, transcribed the interviews, and analyzed the data using the constant comparative method. For EDRS 811: Quantitative Methods in Education Research, I analyzed an existing data set, the School and Staffing Survey, to examine the differences in classroom readiness based on the type of teacher preparation and professional development activities of novice special education teachers. Analysis including the use of the chi squares test, multiple regression, and ANCOVA.
It was not until my fourth semester in the PhD program, however, that I began to truly understand the concept of revealing and concealing in research. Up to that point in my coursework, the studies I designed fit nicely into established methods. Several were simply replications and extensions of previously published studies that I had found interesting. Then I took EDRS 797: Advanced Applications in Mixed-Methods Research. That course forced me to articulate what I believe about knowledge and how we come to know. It also helped me clarify what I believe is critical in research methods and what is researcher preference. The diversity of the studies we read and discussed, as well as the diversity of the students in the class, made me aware of where I stood. As importantly, it expanded my thinking about the possibilities of research design. I was able to begin thinking about how to transition research-based interventions into real-world classroom settings, a primary interest for me, through a series of four assignments aimed at developing such a study (see Focusing Memo, Design Memo, Validity Matrix and Memo, and Data Integration Plan).
The effect of this deeper understanding of creating knowledge was immediately apparent to me in my other coursework. In my EDRS 823: Advanced Research Methods in Single-Subject and Single Case Design, I designed a single-subject study, obtained HSRB approval, implemented the study, and analyzed the results using visual analysis and by calculating effect sizes. The study focused on different methods of teaching letter recognition to a preschool student. When conducting my literature review, I looked beyond the usual special education literature that I had become comfortable with to include brain research studies. It was an interesting challenge to try to integrate findings across fields of research. Additionally, I found that in my analysis I was able to think of a variety of explanations for what occurred, which had been difficult for me in the past. This helped me to consider how I would redesign the study if I conducted it again, and to consider next steps for research in this area.
I also felt the effects of my new understanding of research in my resistance to psychometrics. When I began the class, PSYC 557: Psychometrics, I was frustrated by the emphasis on formulas and very specific procedures for analysis. As the course continued, however, I came to appreciate that psychometrics is similar to all other types of educational research in that it seeks to understand human behavior and strives to make a case for why the conclusions that it reaches are accurate. As with all other types of research, it reveals something and conceals something. It can answer some types of research questions, but not all. For this class, I administered a draft self-efficacy for writing measure, which I had worked on with a colleague for use in our dissertation studies, to about 70 people. I analyzed the results by calculating the internal consistency of the scale, descriptive information for each item, and item-test correlations. Additionally, I conducted an exploratory factor analysis. Again, I noticed my ability to think across different types of research to understand how individuals learn to write. The final product for this project was a technical report on the measure.
On a different note, I also gained research experience conducting a synthesis of meta-analyses of writing intervention research for students with learning disabilities. This writing mega-analysis began as a class assignment in EDSE 841: Intervention Research in Special Education. The focus of the project was to synthesize the results of meta-analyses of writing intervention research in special education. It was a difficult, and sometimes overwhelming, task to undertake in my second semester in the doctoral program. After the course was over, I decided to continue work on the project as an independent study with the goal of writing a manuscript for publication. When I restarted work on the project for my independent study (after completed two more semesters in the program), I saw growth in my ability to understand and evaluate research. For example, it was more clear to me which meta-analyses fit my stated criteria for inclusion and which did not. I expanded the synthesis to include narrative reviews of writing research as well as meta-analyses. In another example, I was able to make decisions about which variables I needed to code from the meta-analyses and reviews and which variables did not provide the information I needed to answer my research questions. My work on the writing meta-analysis continued through my last semester of classes. At that time, I prepared the findings of the synthesis and presented it in a poster session at the Council for Exceptional Children (CEC) conference in Nashville. The growth in my research skills during this extended project can be seen by comparing the project I submitted for EDSE 841 (Table 1 and Table 2) and the poster I presented at CEC.
Finally, several of my courses required me to write applications for HSRB to match the methods section that I wrote for class assignments. The first of these HSRB applications was written during my first semester to go with the quantitative study I wrote for EDRS 810. To complete that first application, I relied on an example the instructor provided of another student’s HSRB application that had recently been approved. I was careful to be sure I included all of the same types of information that were in the example. The following semester, I wrote an HSRB applications for EDRS 812 (a qualitative study). For these, I relied less and less on examples as I became more comfortable writing research methods. By the time I actually had to submit an application to HSRB for my EDRS 823 single-subject study, I felt comfortable with what was expected. I was greatly relieved when the study was approved after I made only a few clarifications.
Conducting Research
In addition to gaining a great deal of knowledge about research through coursework, I have also had the opportunity to practice these research skills through my role as a GRA. In that role, I worked for two years on a multi-year writing grant teaching self-regulated strategy development for writing (SRSD) to middle school students with severe EBD. The first year I joined the team in the middle of a design research study. As the first study in a four-year program of research, the research team was involved in understanding the process of teaching SRSD to students with severe EBD, as well as in measuring the effectiveness of the treatment. As a first-year doctoral student, I learned about the steps in the research process and different data collection techniques. In particular, I was responsible for providing SRSD instruction, maintaining field notes, collaborating with fellow researchers on instructional modifications, conducting post-testing and participant interviews, scoring dependent measures, and compiling and analyzing interview data.
This research project was an important learning experience for me. As I mentioned, one of the most challenging lessons for me to learn was how to think like a researcher. When I stepped into the classroom, it was easy for me to take on the role of teacher. However, I had to learn to view the situation from the lens of research. For example, I learned the importance of closely following lesson plans to ensure fidelity of treatment. As a classroom teacher, I could add or subtract elements from a lesson as I went along, based on student responses. As a researcher, I had to ensure that all elements of the lesson were covered, and that no additional instruction was provided that would confound the results of the study. In another example, I had to learn about evaluating student work based solely on pre-established criteria, rather than on my “teacher judgment.” I did not like the holistic scoring rubric used to evaluate students’ essays because I thought it did not accurately capture the important aspects of our students’ writing. I learned the value of using measures that were established in the literature and provided for a wide range of student responses. Additionally, I learned a great deal about the researcher’s relationship with administrators and teachers at the research site. While we had an open and welcoming relationship with school staff, we had to be careful about the information we shared so that we did not hurt our relationship with teachers or breech the confidentiality of our participants. All of these lessons helped me begin to understand the practical aspects of conducting research.
The second year of the writing study sought to build on the Year 1 design research study by using a more rigorous single-subject multiple baseline design. During the course of this six-month study, I learned a great deal about research design and decision-making. Throughout the study, I participated in research activities similar to those I participated in during the Year 1 study. However, the single-subject design introduced me to new aspects of research. First, we had to obtain multiple baseline measures of students’ writing performance. Because we needed five baseline probes and we knew our students were unwilling writers, we carefully planned out when the probes would be given. I also learned about the use of randomization tests in single-subject research because we randomly assigned start dates to the four instructional groups to allow us to conduct randomization tests on the data.
Second, I gained new skills during the implementation phase of the study. Before beginning at the school site, I was trained to gather behavioral observation data. This resulted in lengthy discussions about our operational definitions of on-task and off-task behavior. Additionally, I trained new researchers to work with students at the research site, and modeled lessons for new instructors. Because we would have four instructional groups rather than one, making sure all instructors provided the SRSD instruction in the same way was critical. Finally, once the study began, a great deal of data was generated very quickly. Establishing systems to collect and store the data was necessary. Our research team used a Blackboard website to house the data, organized by instructional group. As the study progressed, additional data management systems were devised to meet our needs at different phases of the study.
Third, the Year 2 study contained several phases. First, the baseline phase consisted of five essays, as well as other, one-time-only measures. During the second phase of the study, students were taught the POW+TREE SRSD writing strategy for writing persuasive essays. Once that instruction was completed, students completed five post-instruction essays. The third phase of the study was a writing fluency phase. The fluency phase required students to write a one-paragraph persuasive essay using POW+TREE within 10 minutes. After instruction in fluency, students wrote five, 10 minutes essays. Finally, one maintenance and one generalization essay was given 11.5 weeks after fluency was completed. While a multiple-baseline design was the best option given the number of students available and research staff resources, writing so many essays throughout the four-month study proved to be very challenging and frustrating for many of our students. In a follow-up study that same year, we opted for a group design to help alleviate some of the issues we had with students who resisted writing the large number of essays required for the single-subject design.
Lastly, I learned more about scoring and analyzing data during the Year 2 study. During the Year 1 study, three research assistants scored all dependent measures. During the Year 2 study, eight assistants scored measures. This required detailed scoring conventions and an increased attention to maintaining records about scoring. I also learned how inter-rater agreement is determined and reported. As in the previous study, I compiled and summarized information from student interviews. This was a more complex task than in the previous year because of the larger number of students involved in the study and the increased variety of their responses.
Following the single-subject study, we conducted a shorter, quasi-experimental study to teach SRSD to other students at the same school. For this four-week study, we used a crossover design in which one group received two-weeks of SRSD instruction while the second group remained in the control condition (in this case, a basic skills class that was not focused on writing). During the second two-week block of time, the classes switched, with the second group receiving SRSD instruction and the first group in the control condition. The amount of time available to conduct this study was limited due to scheduled high-stakes tests in writing. The time constraint proved to be challenging, particularly given the population of students with whom we were working. Many of the students, who had severe EBD, missed class sessions due to meetings with counselors, in-school suspension, illness, and the like. The limited time frame was also difficult with this population because it takes students with EBD longer to establish trusting bonds with their teachers, which impacts the effectiveness of instruction in the short-term. One benefit of the quasi-experimental design was that collecting dependent measures did, in fact, prove to be easier than it was in the single-subject study. Additionally, data management was easier because there was less data to manage with only one pre-test and one post-test essay per student.
Communicating Research
Another skill that I have gained during the PhD program is the ability to communicate with others about research design, research findings, and what those findings mean for teaching and learning. The first time I had to present about a proposed research study during EDRS 810, I became very nervous when I realized that I forgot to include an important slide in my Power Point presentation. I could hear my voice wavering and my hands shaking throughout the 10-minute presentation. My nervousness surprised me because I had years of experience talking in front of classes as a teacher. Thankfully, I found that the more I learned about research, the more comfortable I became presenting it. Now I am able to talk about research comfortably in front of a variety of audiences. I have presented at CEC conferences and to classes at GMU. Additionally, I have gained experience communicating about research through writing, both for coursework and for publication.
The first presentation I participated in at a CEC conference was during my first semester in the PhD program. It was a poster session on differentiated instruction in content areas. I enjoyed the experience, finding it comfortable to talk about the research with small groups of people in the poster session format. During my second year in the program, I co-presented an hour-long presentation on the topic to a large group. The instructional focus of the presentation made it easier for me to talk about. Also, I knew more about research and felt more confident about being able to understand and answer questions people might have. The presentation went well, which built my confidence for future presentations.
I have also had the opportunity to present the two writing studies I worked on as a GRA. The Year 1 study was presented at the CEC conference in 2009, and the Year 2 study was presented at the CEC conference in 2010 (although I did not give that presentation). In addition, I also participated in another poster session and presentation that focused on our learning about conducting research with students with EBD in alternative school settings. These presentations and poster session were exciting opportunities to share our work with others in the field who were interested in writing.
At this years CEC conference I presented the writing mega-analysis discussed previously. I really enjoyed the experience for several reasons. First, I found that many people were interested in the topic – both teachers and researchers. Second, I had the opportunity to meet other doctoral students and researchers who are interested in writing research, and have been able to correspond with them about it. Third, by presenting what I learned and talking about it with others, I was able to think ahead to next-steps for the project, and what I can do to make it better before submitting it as a manuscript for publication.
In addition to presenting, I have had many opportunities to practice communicating about research through writing. The numerous course assignments that required me to write literature reviews and methods sections have taught me how to include all of the expected information in a paper. I also had the opportunity to write part of a chapter on special education teacher preparation for the upcoming Handbook of Special Education. For that project I explored a new literature base than I had in the past. My portion of the chapter focused on the knowledge new special education teachers must have, which required a different type of writing than the methods sections I had been accustomed to writing for my courses.
Finally, I have had some experience communicating about research to a very different audience – new teachers – through my work as an instructor at GMU. One of my goals when I teach is to help my students become comfortable finding research-based interventions. So I talk about my research with them. I also include assignments that require them to find journal articles on a topic of interest. Through these activities, I hope that they will begin to value what research can tell us about effective instruction, and that they will be able to seek out this information when they are teaching. Because my students are not researchers, there are many opportunities to introduce them to some of the key concepts of research design.
Teaching
Prior to beginning the PhD in Education program at George Mason University, I worked as an elementary school learning disabilities resource teacher for four years, acting as the lead special education teacher and the chairperson of the local screening committee for three of those four years. Additionally, I taught early literacy skills to kindergarteners identified as at-risk for two years, and acted as a mentor resource teacher to first-year special education teachers for three years. Since beginning the PhD program, I have had the opportunity to expand those skills by teaching at the university level. These experiences have taught me that, in many ways, teaching is teaching, regardless of the audience. However, I have also learned to adjust my teaching to meet the needs of my audience, whether they are children or adults, or whether they have special learning needs (e.g., students with disabilities, second language learners).
Teaching in Higher Education
During my time in the PhD program, I have had the opportunity to expand my teaching skills by teaching courses at the university level. I taught EDSE 401/501: Introduction to Special Education during the Summer 2008 and Spring 2009 semesters (see syllabi for Summer 2008 and Spring 2009) During the summer course, I taught five Master-level students. The spring course enrolled 24 students, with roughly equal numbers of undergraduate and graduate students. Course evaluations for overall teaching of the course yielded a mean score of 4.80 for Summer 2008 and a mean score of 4.91 for Spring 2009 (on a scale from 1 to 5).
Teaching the same course over two semesters to two different size classes made up of students at different points in their teacher preparation program was a good learning experience for me. My first, small class consisted mainly of students planning on getting provisional licenses after completing the course. Teaching a small class was comfortable for me and allowed for a great deal of discussion and participation on a daily basis. I felt I had good knowledge of each student’s understanding of course material on a daily basis. When I began teaching a larger class, it was more difficult for me to establish rapport with the group and evaluate student learning throughout a lesson. It was not until a more seasoned professor observed my teaching and provided me with specific feedback that I began to think of ways to vary my teaching to allow for more students to participate through small group activities. Not only did the small group activities allow for more students to respond, but they also allowed for more integration of learning across topics than was possible through a lecture format.
One of the benefits of teaching the same course twice is that I was able to modify assignments to make them more beneficial for student learning. For example, the first time I taught the course, I knew that I wanted students to have to look for scholarly resources to address an instructional topic of interest. I did not want students to merely write article summaries; I wanted them to use the information they found for a purpose. To that end, I developed an assignment where students would have to pick a current issue in the field of special education or an instructional area of interest, and find three articles on that topic. Students used this information to do a 20-minute presentation summarizing the articles and integrating information across articles. While it was not a bad idea for an assignment, the actual implementation of the assignment was confusing for students and the time limit given for the presentation was insufficient for what they were asked to do.
Given my experience with the presentations the first time I taught the class, I modified the assignment for the second class. Students still had to pick a topic of interest, find at least three scholarly resources, and do a presentation. However, this time they had more time for the presentation (30 minutes) and were able to work with a partner. In addition, they had to include a hands-on activity for their classmates to do during the presentation. When students first started doing their presentations, I was concerned because they were not what I had expected. After the first two or three presentations, I made sure I met with students ahead of time and gave more specific guidance about what should be included in their particular presentation. I was pleased that the quality of the presentations improved greatly. By the end of the semester, many of the presentations were outstanding. That said, in the future I would modify this assignment again. Thirty-minute presentations took up a great deal of class time, particularly with so many students in the class. Perhaps in the future I will give students more options for the assignment, like making a You Tube video, a web page, or a printed resource related to their topic.
Another lesson I learned teaching at the university level was how to deal with students who are not making expected progress. The first semester I taught the course, I had a student who turned in assignments late and incomplete, and was consistently late to class. I met with her one-on-one on several occasions to make plans for completing the work with specific tasks for her to accomplish by certain deadlines. She consistently failed to meet those deadlines. In the end, she failed the course. In reflecting on that experience, I realized that I should have handled it differently. One thing I should have done differently was to address the issue the day she failed to turn in the first assignment on time, rather than waiting two extra days for her to turn it in. Another thing I should have considered was talking to her about whether or not she should drop the course when it became clear that she was rapidly falling behind. The second semester I taught the course I did much better at addressing issues immediately. For example, one student did not do her field experience project correctly. After reading her paper, I immediately emailed her and was clear about exactly what she needed to do and the timeline for completing the work. She emailed me back right away with the specifics of how she would get the work done, and she followed the plan. These are just some of the lessons I learned about teaching students at the university level.
In addition to teaching Introduction to Special Education, I was also a guest speaker for two other classes. For EDCI 545: Assessment and Differentiation my presentation focused on differentiation for students with special learning needs. This presentation required me to combine my experience as a co-teacher in elementary school classrooms and my knowledge of research-based instructional strategies to teach pre-service teachers strategies for reaching diverse learners in their classrooms. Specifically, my co-presenter and I focused on characteristics of students with disabilities, direct instruction and strategy instruction in the core curricular areas, and behavior management and social skills strategies.
As a guest speaker in EDSE 540: Characteristics of Students Accessing the General Education Curriculum, I gave a presentation about the special education process and IEPs. The presentation began with a description of the special education process from referral to eligibility to reevaluation. I also discussed the essential components of the IEP, walked through a typical IEP meeting, and conducted an IEP goal writing activity with students.
Teaching in Elementary and Secondary Education
I am a certified special education teacher in the Commonwealth of Virginia. For four years I worked as a learning disabilities resource teacher in an elementary school, serving students with a variety of mild disabilities in grades 1-5. Additionally, for two years I provided early literacy intervention to kindergarteners who entered school with low pre-literacy skills. In this position, I worked with a large population of students who were low-income, English language learners.
My experiences working on writing intervention research projects has afforded me an opportunity to expand my teaching skills to teach middle school students with severe emotional and behavioral disorders (EBD). Working with this student population was very different from working with elementary-aged students with mild disabilities. My younger students with mild disabilities were generally well behaved and compliant. They were typically enthusiastic about learning and willing to work hard to learn new skills. In contrast, the middle school students with severe EBD that I worked with did not want to complete writing tasks, were resistant to new teachers, frequently missed class sessions, and progressed more slowly with their writing skills. One of the most important lessons I learned about working with students with EBD was that they need to have a trusting relationship with their teacher in order for them to be willing to engage in academic work. Students are more willing to trust teachers who are fair (i.e., consistent in their expectations across students and from day-to-day), who make an effort to get to know them as individuals, and who allow students to start each day with a “clean slate.”
Teaching Teachers
Prior to beginning my PhD studies, I also worked for three years as a mentor resource teacher. In this position, I worked with first-year special education teachers, meeting with them each week and working on everything and anything with which they needed help. The teachers I worked with ranged from preschool to third grade teachers, and taught students with moderate to mild disabilities. Working with beginning teacher so intensively over the course of the school year taught me a lot about teacher development and the needs of new teachers. For example, ever year I would have a teacher say to me, within the first few weeks of school, “I need help with math (or reading or writing).” I would teach them how to collect data about where their students were currently performing, and help them develop both unit plans and individual lesson plans. I often modeled lessons for them. In spite of this intensive help, instruction did not improve. Then, around January or February, the same teacher would tell me that she needed help with math. Again, we would talk about assessment, planning, and instruction. This time, however, the teacher would make changes that benefited her students. This taught me (again and again) that teachers develop in stages, and the support they are given needs to match where they are developmentally. This experience made me interested in the ways in which teachers are prepared, including their preparation during the first few years of teaching, and ways it can be improved.
Service
While in the PhD program, I have provided service to peers with in the PhD program, other students within the College of Education and Human Development, and to the field of special education more generally. Within the PhD program I am a peer mentor to two PhD students majoring in special education. Additionally, I have provided assistance to my peers with their research projects by scoring dependent measures and fidelity of treatment measures, or by helping them revise papers. Within the College of Education and Human Development, I am a faculty mentor to a student in the BIS program who is combining coursework in psychology and special education. As part of my role as a mentor, I am assisting her with a culminating research project in her area of interest.
Finally, I provide service to the field of special education through my role as the Assistant Editor for Exceptional Children. In this position, I have gained many new skills, as well as increased my understanding of high quality research design and the publication process. I became the assistant editor when Dr. Mastropieri and Dr. Scruggs became the new editors of the journal in August 2009. Therefore, I was involved in the transition as the journal moved from one editor to the next. This required me to coordinate with the previous assistant editor to make sure all submitted manuscripts were received and reviewed in a timely manner. I also set up our internal data management systems to track manuscripts, reviewers, and due dates. Additionally, I compiled the list of editorial board members and collected contact information for all reviewers. During the transition time, I also aided in making sure the new electronic submission portal was functional and met the needs of the editors and reviewers.
One of my main responsibilities as the Assistant Editor is to manage manuscripts submitted to the journal. This requires keeping track of submission dates, maintaining records of reviewers assigned to referee articles, making sure reviews are completed in a timely manner, and documenting editorial decisions. I also code each article and maintain an SPSS file of data related to the type of article, sample description, and so on.
In addition to data management, I also support authors and reviewers throughout the submission and review process. For example, if authors have difficulty with the new electronic submission portal, I can help them with the process or find answers to their questions. I also maintain contact with reviewers in case they have any questions or need support during the review process. This support can include activities like resending articles for review, reminding reviewers that reviews are overdue, or answering questions about the website.
My role as the assistant editor of the journal has also deepened my understanding of high quality research design and data analysis. When coding manuscripts, I can often tell when a study does not meet high research standards because of the lack of information provided on certain aspects of the research design. Additionally, I have noticed that the experts in the field of special education who review articles expect manuscripts to meet very exacting standards. Through my work on the journal, I will have the opportunity to review manuscripts and compare my evaluation with that of other reviewers, which will be an excellent, “real-world” learning opportunity for me.
All in all, I have seen a great deal of growth in my research and teaching skills during my time in the PhD program. It is exciting to see how all of these abilities have come together as I prepare to do my dissertation. I know that I will continue to grow during this process and that I will be prepared to meet the demands of a career in higher education when I graduate.
Site Design and Map
smills5@gmu.edu
